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ABSTRACT 

During the 1985-86 school year, 16 North Carolina 
school units began a four-year pilot program that used a variety of 
incentives, evaluation strategies, and staff development activities 
to improve teachers » skills, and, indirectly, the achievement of 
students. After three years of the Career Development Program (CDP) , 
the California Achievement Test scores of students in the school 
units were analyzed to see if their teachers* career development 
program made a difference. A control group of 15 units comparable to 
the CDP group also was analyzed in order to clarify which effects 
might be attributable to career development. The test showed more 
improvement in the CDP units than in the match units. For example, at 
grades 3, 6, and 8, the number of CDP units scoring below the 
national median declined, whereas most match units continued to have 
scores below the median. In addition, more than 63 percent of the CDP 
units experienced growth in each grade over the period, whereas in 
the match units, the corresponding percentage never exceeded 50 
percent. Improvements in achievement were more widespread in the CDP 
units. Finally, sixth-to-eighth grade scores improved for the CDP 
units more than for the control units. The study concluded that the 
CDP units* students appear to have made more substantial gains faster 
than did other students, in their region anfl. qtate. (KC) 
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STUDENT ACHIEVEMENT IN CAREER DEVELOPMENT 



PROGRAM PILOT UNITS, 1985-88 



EXECUTIVE SUMMARY 



in school year 1985-86, sixteen North Carolina school units agreed to 
participate in a four-year pilot program intended "to attract and retain the 
best people to education". Using a varie^ty of incentives, evaluation strategies, 
and staff development activities, the Program hoped to improve teachers' skills 
and, indirectly, the achievement of students. Ihis report analyzes the students' 
achievement in Grades 3, 6, and 8 over the three years that the Program has been 
in place. Using unit averages from the California Achievement Test Total Battery, 
we will analyze these students* performance. 

In order to clarify which effects might be attributable to Career Development, 
a second group of achievement scores will be examined. A nr:atch group of 15 units 
that are comparable to the CDP group on four dimensions was created. The four 
variables are average daily membership, per-pupil expenditure (excluding school 
food service), percentage of students planning to attend college^ and geographic 
location in North Carolina. By comparing this group* s achievement to that of 
CDP units, patterns of improvement will be seen in CDP units that in the match 
units are more randomized and generally less positive. 

Essentially, the analysis found the following: 

0 At Grades 3, 6, and 8, the number of CDP units scoring below 



the national median declined. In the match units, the 



number declined only for Grade 3. In fact, the number of 



units below the median in Grade 6 increased. 



ERLC 





^ More than 63% of the CDP units experienced growth in each 

grade over the period. In the match units> the corresponding 
percentage never exceeded 50%. 
• Improvements in achievement were more widespread in the CDP 
units, affecting low-achieving and high-achieving units. By 
comparison, while low-achieving match units often made gains, 
high-achieving units often lost ground, 
t If the 1986 6th graders' performance is compared to their 

performance two years later (1988 8th grade), we see gains in 
both the CDP and the match units among the lowest scoring sixth 
grades. However, match units in the middle and upper range were 
more likely to decline in achievement, and more likely to show 
larger declines than CDP units. 
When compared on virtually any measure, CDP units, individually and as a grou^ 
experienced more positive change in student achievement than did the matched unit 
While North Carolina's students generally improved over the period, the CDP units' 
students appear to have made more substantial gains faster than did other students 
in their region and state. 
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In school yodt 1985-86, sixteen school units launched a pilot proyrani designed 
to attract and retain good educators. While improved instruction and consequently 
improved learning were clearly goals of the pilot program, the interventions 
directly affected teachers, not students. Whereas curriculum changes directly 
influence, or seek to influence, the learner, the Career Development Program 
aims at instruction - or teacher behaviors - as the locus of change. While 
the effect on student achievement is desired, it is not directly traceable to 
CDP features. 

This indirect relationship between strengthening instruction and strengthening 
student performance is further complicated by the fact that the specific teacher 
behaviors that are encouraged are themselves correlative, not causal, of improved 
student achievement. In the research that underlies North Carolina's performance 
appraisal system, all of the teaching skills are correlated powerfully with student 
achievement gains. It is, therefore, reasonable to ask: What effects has the Career 
Development Program pilot had on student achievement in the 16 participating districts 
This report seeks to answer that question. 

Frankly, we undertake this analysis with some reluctance. Throughout the 
pilot period, sjperintendents , principals, and teachers have reported almost 
universally that teaching has improved and that student performance has improved. 
In 1986, we reported that 57% of more than 4,000 teachers responding to a DPI 
survey agreed that ^^participating in the CDP has helped me perform my role more 
effectively". (N. C. Board of Education, 1986). Only 2Vl> of respondents disagreed 
with the statement. 



In addition, 43% of respondents agreed that Effective Teaching Training and 
the new evaluation system had imprrved the quality of instruction in their 
schools. An additional 28% weren't sure. Even at that early date, 41% of 
respondents agreed that the Career Development Plan was likely to influence 
«ducdtiori in. a positive way. Of all respondents, only about one-third (36X) 
disagreed with that statement. 

Moreover, in a survey conducted by the North Carolina Association of 
Educators at about the same time (NCAE, 1986), 37% of respondents felt they 
had become better teachers and that classroom instruction had somewhat improved. 
Another 13% felt that they had definitely become better teachers. A quarter felt 
there had been no change, and only 17% felt classrocn" performance had suffered. 
As recently as Spring, 1988, in a second NCAE survey, (Bunche, 1988) 58% of the 
respondents agreed that, "lessons based on the 6-step lesson plan have improved 
student Irarning", while 64% agree that "observation and evaluation have helped me 
to improvo specific aspin'ts of my teaching". 

Our reluctance doi".. not stem fi n <iny I ear that student dcliievumeiit has 
suffered because of the Career Development Program pilot. Rather, we hesitate 
to conduct the analysis because of the complex interaction between teacher in-puts 
and student outcomes. Teaching and learning are far too complex to reduce to a 
single number. Much of what is taught and learned goes untested. That is in the 
nature of evaluations which are £er force limited in what can be measured. 

Moreover, students experience far more than is taught by teachers. Put 
another way, teachers exercise too little control over all their students' 
ftxperienco for studonl achievement tests to I'e valid in discovering the relation- 
ship botWfM>n any sin(jle t.e<uhi;»'s eirurl;, and any _si_ri(j_1_e studenl's achievement. 
However, il we consider unit average test scuros, and if wo consider this analysii. 
as programmatic evaluation, much of the difficulty disappears. 
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We will still not factor out all the interference between teaching nnd 
learning, but by aggregating the data at the unit level and then comparing units, 
we will gain a useful sense oF change in performance. Moreover, if we find trends 
or patterns in the data, these will be highly suggestive. With these limitations 
in mind, then, we proceed to the actual analysis of the data. We do not believe 
that these data wilV indicate causal relationships between Career Development 
effects and student achievement. We do, however, believe that we will see correla- 
tions between participation in Career Development and improved student achievement. 

We cannot attribute all positive change to Career Development, but, if we 
fail to discover similar patterns of achievement in a matched sample of students, 
then clearly some effect of Career Development is present and is influencing 
students' achievement. 
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METflODOLOGY 

In order to assess the impact of Career Development on student achievement, 
we analyzed the results of the California Achievement Tests (CAT), which are 
administered annually to children in North Carolina's public schools. Because 
of the availability of data, we focused on the performance of children in Grades 
3, 6. and 8 during school years 1985-86, 1986-87, and 1987-88. (Div. of Research, 
1986). Because the CAT was re-normed in 1985-86, comparisons prior to the pilot 
period were not attempted. We also collrctod and analyzed performance data for 
yoursgsters in 15 additional units. We hoped, by using these comparison analyses, 
to be able to isolate patterns in the performance data that could not be attributed 
to chance, to other reform efforts -- most notably the Basic Education Program and 
the Standard Course of Study -- that have been undertaken in North Carolina schools, 
nor to other causes than CDP. The match units were selected by Dr. Carol Furtwengler 
of the Research & Service Institute of Nashville, Tennessee. Dr. Furtwengler is the 
third-party evaluator retained by the N. C. General Assembly to study CDP imple- 
mentations. She selected the match units on the basis of geographic distribution, 
average daily membership in the match units, per-pupil expenditure (excluding 
school food service), and percentage of students planning to attend college. 
(Private communication from Carol furtwengler, 1988). Thus, Dr. Furtwengler 
constructed a mirror-image of the CDP pilot units. Similarities and differences 
in performance profiles in the pilot and match units should, therefore, allow us 
to draw some tentative conclusions about the effects of Career Development. Since 
the match units did not voluntarily participate in this study, we will refer to 
them by letter (District A, etc.) in order to avoid any hint of criticism of the 
performance of their students or teachers. (Research and Testing Services, 1987, 
Division ol Testing, 1988). 
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RESULTS 

The CAT scores for third grade students in the pilot units are presented 
in Table 1. For convenience, we have selected the median national percentile 
average on the total battery for each unit as our reported figure. For each 
year, we have arranged the units in rank order, followed by the unit's average 
score. The number in parentheses following the 1987 score indicates the amount 
of changu. rollowing thp 1988 averdge, we present two numbers in parenthesis. 
The first shows the change between 1087 and 1988. The second indicates the 
aiiiounl of change between 1985-86 and 1987-88, the entire pilot period. The 
plus sign (+), of course indicates gain, while the minus (-) indicates loss. 
The dark line within the columns separates units scoring above the 50th per- 
centile, or national average, from those units scoring below the national average. 

G rade 3 Resul ts 

Examination of Table; 1 makes several things clear. First, the number of 
units scoring below the bOth percentile declined steadily over the period from 
four to two to one in 1987-88. Moreover, both the ceiling (highest score) and floor 
(lowest score) rose over Lhe period, when the units are taken as a whole. For 
individual units, the averages increased in 12 units over .the period, remained 
unchanged in one unit, and declined in three units. In the units experiencing 
decline over the period (Roanoke Rapids, Harnett, and Mecklenburg), however, two 
of them had improved over their 1986-87 performance. 

The gains made by the four units performing below the 50th percentile in 
l<JHl.-«f) should not be overlooked. With only one exception, these units all moved 
up significantly over the period, with Perquimans registering a dramatic 12 per- 
centiif point increase. 
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TABLE 1 

3rd Grade CAT Scores (Unit Average) 



for CDP Units 



1986 




1987 






Burl ington 


76 


Burlington 


73 


(- 3) 


Haywood 


71 


Tarboro 


70 


(+ 9) 


R. Rapids 


67 


R. Rapids 


68 


(+ 1) 


Buncombe 


67 


Haywood 


68 


(- 3) 


Harnett 


64 


N. Hanover 


67 


(+ 6) 


■- 


64 


Buncombe 


65 


(- 2] 


N. Hanover 


61 


Burke- 


63 


(- 1) 


Tarboro 


61 


Perquimans 


62 


(+14] 


Mecklenburg 


60 


Alexander 


59 


(N.C. 


Orange 


59 


Montgomery 


59 


(+ 7) 


Alexander 


59 


Mecklenburg 


57 


(- 3] 


Montgomery 


52 


Orange 


57 


(- 2] 


Perquimans 


48 


Harnett 


56 


(- 8] 


Greene 


47 


Salisbury 


52 


(+ 8] 


Sal isbury 


44 


Greene 


46 


(- J] 


Chowan 


43 


Chowan 


39 


(- 4] 



Burl ington 


77 


(+ 4; 


1) 


Haywood 


71 


(+ 3; 


,N.C) 


Alexander 


71 


(+12- 


,+12) 


Perquimavjs 


70 


(+ 8: 


.+22) 


Burke 


69 


(■>- 6- 


,+ 5) 


N. Hanover 


G8 


(+ 1- 


.+ 7) 


Buncombe 


68 


(+ 3- 


.+ 1) 


Tarboro 


67 


f- 3 


;+ 6) 


R. Rapids 


63 


(- 5 


4) 


Orange 


62 


(+ 5 


;+ 3) 


Harnett 


60 


(4 4 


4) 


Mecklenburg 


58 


(+ 1 


2) 


Montgomery 


56 


(- 3 


,+ 4) 


Greene 


•15 


(+ 9 


,+ 8) 


Chowan 


50 


(+11 


.+ 7) 


Sal isbury 


47 


(- 5 


;+ 3) 



6 

n 



Moreover, whereas only two units scored above the 70th percentile in 1986, by 
1980 four units had moved into this range. Put r^other way, units in all parts 
of the range were improving performance. Table ? shows the difference per year 
in the range of scores and the distribution of units to deciles. 

In £un,n.ary,. then, 75% of the CDP units' shewed improvements over the period, 
in perfurntance of 3rd grade students, while 19'^ of the units showed decline, with 
one unit remaining unchdnged. Overall, both high-arid low- achieving units experienced 
improvement, with only one unit remaining below the 50th percentile at the end of 
the period. Tables 3 and 4 display the same information relative to the match 
units. It is clear that achievement in the match units, as a group, is somewhat lower 
than the pilot units throughout the period. Four of the match units are below the 50th 
percentile in 1985-86 and 1936-87, while three remain below the 50th percentile in 
19f7-F8. All three of these units have been below the 50th percentile throughout 
the period, althouoli the floor rises from the 31st to the 35th percentile. 
Moreover, the ceiling rises dramatically from 64 to 72, but this is the result o+ 
the spectacular gain of Unit L, which moved from 49 to 72. The natural ceiling 
rises only to 68th percentile, a steady but not spectacular increase. 

Overall, eight units witnessed improvement in 3rd grade, while seven ex- 
perienced declines. The four units that began below the 50th percentile all 
experienced gains but, except for Unit L and to a lesser degree Unit M, the gains 
were slow and uneven. Two of the four, for example, experienced decreases in 
1988 from 1987 levels. As Table 4 shows, the range increased over the period, 
indicpting that growth at the top is occurring faster than growth at the bottom 
of the qroup. Howevet-, if we correct for Unit L, there is no change in the size 
of the range. 



.12 



TABLE 2 

Range and Decile Distributions of CDP Units Per Year 

Range Deciles : Units 

1986 43 - 76 (33 points) 30th: 0; 40th: 4; 50th: 3; 60th: 7; 70th: 2 

1987 39-73 (34 points) 30th: 1; 40th: 1; 50th: 6; 60th: 6; 70th: 2 

1988 47-77 (30 points) 30th: 0; 40th: l;50th: 4; 60th: 7; 70th: 4 
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lABLE 3 

3r(l Grade CAT Scores (Unit. Average) tor "atch Unil.;; 



1986 



1987 



A* 


64 


C 66 


(+ 3) 


L 72 


(+10;+23) 


B 


64 


D 65 


(+ 4) 


C 68 


(+ 2;+ 5) 


c 


63 


B 62 


(- 2) 


J 66 


(+ 7;+ll) 


D 


61 


L 62 


(•♦13) 


B 62 


(N.C.;- 2) 


t 


ro 






u Oc. 


( + &•+ 9) 


F 


59 


A 58 


(- 6) 


D 58 


(- 7; -3) 


G 


58 


E 58 


(- 2) 


M 57 


(+15;+ 9) 


H 


57 


H 56 


{- ]) 


A 57 


(- 1;- 7) 


T* 
I 


00 




(- 31 


K 56 


f + 1 •+ 2) 


J* 


55 


K 55 


(+ 1) 


F 56 


(+ 4;- 3) 


K* 


54 


F 52 


(- 7) 


G 54 


(- 1;- 4) 


L 


49 


N 46 


(+ 3) 


H 54 


(- 2;- 3) 


M* 


48 


I 43 


(-13) 


I 45 


(+ 2;-ll) 


N* 


43 


M 42 


(- 6) 


N 44 


(- 2;+ 1) 


0* 


31 


0 36 


(+ 5) 


0 35 


(- 1;^- ^) 



* Units participated in TPAS Pilot (1985-87) 
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TABLE 4 

Range and Decile Distributions of Match Units Per Year 








Range 


Deciles : Units 






1986 


31 - 64 (33 points) 


30th: 1; 40th: 3; 50th: 6; 60th: 5; 


70th: 


0 


1987 


36 - 66 (30 points) 


30th: 1; 40th: 3; 50th: 7; 60th: 4; 


70th: 


•0 


1988 


35 - 72 (37 points) 


30th: 1; 40th: 2; 50th: 7; 60th: 4; 


70th: 


1 
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In summary, then, the match units as a group show some of the tendency towards 
improvm-nt seen in Llic- CDP units. (See liyute 1). The growth, however, 
Appears to le slower. Ai. t:naiysis of the individual units makes clear, the growth 
is also more randomized. While 75% of the CDP units experienced growth over the 
period, only slightly more than bO% of the match units experienced growth. Taken 
together, the match units show no clear patterns, unless one considers the fact 
that Units J, K, M, and 0 all experienced gains over the period and were involved 
in the UPI-sponsored pilot of the Teacher Performance Appraisal System in 1985- 
1987. The meaning, however, of this pattern is not clear, especially in view of 
the rathr;» unfortunate performances of Units A and I, who also participated in the 
TPAS pilot. 



Grade 6 Results 

Tables 5 and 6 present the same information for Grade 6 in the CDP pilot units. 
Generally speaking, achievement test scores for 6th grade students are lower than 
they are for 3rd. grade students for several reasons, mostly having to do with the 
greater curricular freedom experienced in Graaes 4, 5, and 6 and the increased 
likelihood that the test and the curriculum will diverge. We see this phenomenon 
clearly in Table 5 as compared to Table 1. However, Table 5 shows similar patterns 
ot achievement in other respects. 

\r, 6th grade for CDP unit", as for 3rd grade, we see that the number of units 
ending below the 50th percentile declines, although less dramatically. It is 
noteworthy, however, that the floor rises rapidly and steadily from the 33rd per- 
centile to the 42nd. While the ceiling also rises from 69 to 71, it is interesting 
that in VJiib only three units were above the 60th percentile, but this number increases 
to 5 in .988. Of the pilot units, 11 experience improvements over the period, one 
has no change, and four experience declines. 
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TABLE 5 

6th Grade CAT Scores (Unit Average) for CDP Units 



1986 IMZ 1^ 



burlincton 


69 


Burlington 


72 (+ 3) 


Burl ington 


71 (- 1;+ 2) 


Haj?wo6d 


63 


Haywood 


66 (+ 3) 


N. Hanover 


66 (+ 4;+ 8) 


R. Rapids 


GZ 


R. Rapids 


65 (+ 3) 


Buncombe 


63 (+ 2;+ 5) 


Burke 


59 


N. Hanover 


62 4) 


Burke 


62 i-i- 5;+ 3) 


N. Hanover 


58 


Buncombe 


61 (+ 3) 


Haywood 


62 (- 4;- 1) 


Bu ncombe 


5^. 


Alexander 


58 (+ 1) 


Chowan 


59 (+10;+ 5) 


I'ecklenbuig 


57 


Burke 


57 (- 2) 


Alexander 


57 (- 1;N.C.) 


Al exa/. J'er 


57 


Mecklenburg 


55 (- 2) 


R. Rapids 


56 (- 9;- 6) 


Chowan 


54 


Salisbury 


54 (+16) 


Mecklenburg 


56 (+ 1;- 1) 


Orange 


53 


Orange 


54 (+ 1) 


Orange 


52 (- 2;- 1) 


Harnett 


50 


Perquimans 


53 (+ 8) 


Harnett 


51 (+ 2;+ 1) 


Tarboro 


48 


Montgomery 


50 (+ 6) 


Montgomery 


50 (N.C.;+ 6) 


Perquimans 
Montgomery 


45 


Tarboro 


50 (+ 2) 


Perquimans 


50 (- 3;-!-/'6) 


44 


Chowan 


49 (- 5) 


Tarboro 


49 (- l;>v 1) 


Salisbury 


38 


flarnett 


49 (- 1) 


Salisbury 


42 (-12;+ 4) 


Greene 


33 


Greene 


39 (+ 6) 


Greene 


42 (+ 3;+ 9) 
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TABLE 6 

Range .and Decile Distributions of CDP Units Per Year 

Range Deciles : Units 

1986 33 - 69 (36 points) 30th: 2; 40th: 3; 50th; 8; 60th: 3; 70th: 0 

1987 39 - 72 (33 points) 30th: 1; 40th: 2; 50th: 8; 60th: 4; 70th: 1 
iS88 12 - 71 (29 points) 30th: 0; 40th: 3; 50th: 8; 60th: 4; 70th: 1 
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In prjly one case, however, is the decline larger than 1 percentile point. Tho 
iruproveinent scores range as high as ^9 for one unit, With one at *8, one at ^6, 
three at +5, one at +4, one at +3, one at +2, and two at +!• 

Again, as in the 3rd grade profile, we find a general pattet^n of improvement 
among the pilot units, with the bottom units moving up steadily and rapidly. Two 
r.nits lost ground in both 3rd and 6th grade. However, ten units experienced im- 
provement in both 3ra and 6th grades over the period 1985-88. When the data in 
Tables 7 and 8 are examined, these patterns do not emerge. 

Tables 7 and 8 present the achievement data for Grade 6 in the match units. 
The general level of achievement for this group starts out lower than for CDP units 
and declines over the period, unfortunately. While the ceiling remains constant at 
63, the floor drops from the 37th percentile in 1986 to the 32nd in 1988. Moveover, 
more units fail to attain the median (50th percentile) in 1988 than in 1986 or ^ 
1987. Among individual units, 33% experience increases, two register no change 
over the period, and eight experience declines, with half of these experiencing 
declines of more than 5 percentile points. Of the five units with the lowest 
scores in 1986, tv/o are worse off in 1988. Moreover, two units (L and G) begin 
the period in the 5th decile, but decline by 1988 to the 49th percentile. Of nine 
units scoring below the 50th percentile in 1988, fully two-thirds experienced 
declines over the period. Only two (H and 0) improved their scores. The TPAS 
pilot experience, incidentally, appears to have had only mixed success. Two of 
the pilot units (I and J) experienced declines; two (0 and K) made improvements 
and one (M) experienced no change. 

As Table 8 demonstrates, the distance between the lowest and highest scaring 
units increased over the period, caused by the decline in the low performers. 
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TABLE 7 

6rd Grade CAT Scores (Unit Average) 



1986 1987 



A* 


63 


A 67 


(+ 4 


C 


63 


r 


ul. 


(- 1 


L 


56 


E 


61 


(+ 7 


B 


56 


B 


61 


(+ 5 


J* 


55 


J 


56 


(+ 1 


E 


54 


L 


56 


(N.C 


G 


52 


G 


55 


(+ 3 


F 


48 


H 


51 


(+ 5 


D 


48 


F 


47 


(- 1 


M* 


, 47 


. D 46 


(- 2 


H 


46 


K 44 


(+ 2 


N* 


43 ■ 


M 


41 


(- 6 


K*- 


42 


I 


41 


(+ 2 


I* 


39 


N 


39 


(- 4 


0* 


37 


0 


31 


(- 6 



* TPAS Pilot Units 1985-87 



for Match Units 





1988 




A 63 


(- 4 


;N.C.) 


C 62 


(N.C 


+1) 


E 62 


(+ 1 


.+ 8) 


B 58 


(- 3 


► + 2) 


J 54 


(- 2- 


1) 


K 50 


(+ 6- 


,+ 8) 


G 49 


(^ 6; 


- 3) 


L 49 


(- 7- 


7) 


H 48 


(- 3; 


+ 2) 


M 47 


(+ 6- 


,N.C.) 


D 44 


(- 2; 


- 4) 


F 42 


(- 5; 


- 6) 


0 41. 


(+10; 


+ 4) 


N 38 


(- 1; 


- 5) 


I 32 


(- 9; 


- 7) 



1 





TABLE 8 

Range and Decile Distributions of Match Units Per Year 










P%ange 


Deciles 


: Units 








1986 


37 - 63 (26 points) 


3pth: 2; 


40th: 6; 50th: 5; 60th: 


2; 


70th: 


0 


1987 


31 - 67 (36 points) 


30th: 2; 


40th: 5; 50th: 4; 60th: 


4; 


70th: 


0 


1988 


32 - 63 (31 points) 


30th: 2; 


40th: 7; 50th: 3; 60th: 


3; 


70th: 


0 
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If any pattern emerges from this data, it is that the poor performers are 
performing more poorly at the end of the period. (See Figure 2). Of eight 
units below the median in 1986, four lose ground by 1988 and only one n5.es to 
the national median of 50. When the data in Table 7 are compared with ihose 
in Table 3, it becomes clear that Units E, K, and 0 experienced growth in both 
3rd and 6th grade over the period. However, Units D, F, G, and I ^:ysz ground in 
both 3rd and 6th grades. Compare this with the ten units in the COP group that 
improved both 3rd and 6th grade performance, while two experienced decline. 
Clearly, a very different pattern emerges whether the analysis is grade for grade, 
year for year, across the period, or for component units in the two groups. 

Grade 8 Results 

The data presented in Tables 9 and 10 concern 8th grade achievement in CDP 
pilot units. At first glance, they appear somewhat troublesome. For the first 
time, we see the range of difference in CDP units expanding, signifying that the 
ceiling is moving up faster than the floor or that the floor is actually dropping. 
However, a second look that considers only 15 of the 16 units is' reas:uring. The 
same patterns of achievement seen earlier are confirmed. While the Greene County 
data cannot be ignored, they seem to create a pattern of only one. The number of 
units scoring below the 50th percentile in 1988 is only half of those below the 
50th in 1986. Moreover, of the four lowest scoring units in 1988, only Greene is 
actually losing ground and one unit is improving while two stay steady. Overall, ten 
units experienced score increases during the period, while three units declined. 
Five units experienced increases of +4 or better. Thus, improvement was general- 
ized among high, medium, and low scoring units over the period. This is in marked 
contrast to the performance of 8th grade students in the 15 match units. Tables 
11 and 12 present the data for these students. 
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TABLE 9 

8th Grade CAT Scores (Unit Average) for CDP Units 



1986 




1987 












R. Rapids 


05 


Burlington 


64 


(N.C.) 


Burlington 


68 


(+ 4;+ 4) 


Burl ington 


64 


Haywood 


62 


(+ 1) 


R. Rapids 


65 


(+ 4;N.C) 


Haywood 


61 


Buncombe 


62 


(+ 4) 


Haywood 


62 


(N.C.;+ 1 


Buncombe 


58 


R. Rapids 


61 


(- 4) 


Chowan 


57 


(+10;+ 7) 


Tarboro 


54 


N. Hanover 


55 


(+ 4) 


N. Hanover 


57 


(+ 2;+ 6) 


Sal isbury 


53 


Perquimans 


53 


(+ 2) 


Buncombe 


56 


(- 6;- 2) 


Alexander 


52 


Montgomery 


52 


(+ 3) 


Mecklenburg 


55 


;+ 4;+ 4) 


Perquimans 


51 


Harnett 


51 


{+ 2) 


Alexander 


55 


(+ 6;+ 3) 


N. Hanover 


51 


Mecklenburg 


51 


(N.C.) 


Salisbury 


54 


(+ 9;+ 1) 


Mecklenburn 


51 


Burke 


51 


{+ 1) 


Bui'ke 


53 


(+ 2;+ 3) 


Chowan 


bO 


Orange 


.50 


{+ 1) 


Harnett 


53 


[+ 2;+ 4) 


Burke 


50 


Alexander 


49 


(- 3) 


Tarbo^'o 


52 


(+ 3;- 2) 


Montgomery 


49 


Tarboro 


49 


{- 5) 


Orange 


52 


(+ 2;+ 3) 


Harnett 


49 


Chowan 


47 


\~ ^) 


Perquimans 


51 (■ 


- 2;N.C.) 


Orange 


49 


Sal isbury 


45 


(- s) 


Montoor.:ery _ 


49 


(- 3;f;.C. 


Greene 


40 


Greene 


38 


(- 2) 


Greene 


36 


(- 2;- 4) 



TABLE 10 

Range and Decile Distribi.-tions of CDP Units Per Year 

Range Deciles : Units 

1986 40 - 65 {25 points) 30th: 0; 40th: 4; 50th: 9; 60th: 3; 70th: 0 

1987 38 - 64 (26 points) 30th: 1; 40th: 4.; 50th: 7; 60th: 4; 70th: 0 

1988 36 - 68 (32 poir.ts) 30th: 1; 40th: 1; 50th: 11; 60th: 3: 70th: 0 
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Table 11 shows that both the group floor and ceilings rose over the period. 
In general, then, achievement improved for the group. Unfortuna{.ely, as many 
units failed to achieve at or above the national median 1n 1988 us in 1986, 
although four of these low- scoring units marked improvements over the f^eriod. 
For the most part, units below the median in 1986 were still there in 1988, with 
the notable exception of Units J and K, which changed places. Three of the top 
four units experienced improvement over the period while the middle units lost 
ground or showed no change. 

Overall, these units did not do as well, relatively or absolutely, as did 
the CDP units (See Figure 3). The match group had a tighter range throughout the 
period as compared to the CDP units, partly because the match group's ceilinc 
' er eoualed the CDP ceiling and because the match group's floor ro'^e. However, 
'ndividual units did not do as well as the individual CDP u. its. Only eight 
of the matched units made improvements over the period, but four lost ground. 

CveraV{ Results 

An overview of the net changes over the period at each grade level for each 
unit is provided in Table 13. While eight of the match units made gains in 3rd 
grade, 12 of the CDP units experienced improvements. At the 6th grade level, only 
one third of the match group made gains, while 11 of the CDP units improved. 
Among the match units, eight experienced gains in 8th grade compared with ten of 
the CDP units. Moreover, five CDP units — Chowan, New Hanover, Salisbury, Burke 
and Burlington — made gains at each grade level, while no unit lost at all levels. 
In the match units, E, K, and 0 experienced improvements at all levels. However, 
Units F and G experienced losses at each grade level. 
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TABLE 11 

8th Grade CAT Scores (Unit Average) 



1986 1987 



c 


59 


B 


58 


(+ 3) 


6 


55 


C 


56 


(- 3) 


G 


55 


G 


55 


(N.C.) 


E 


54 


K 


55 


(+ 6) 


A 


53 


E 


54 


(M .C. ) 


0* 


52 


J 


53 


(+ 1) 


H 


51 


L 


52 


(+ 3) 


L 


49 


H 


50 


(- 1) 


K* 


49 


A 


49 


(- 4) 


M* 


45 


D 


48 


(+ 3) 


f 


45 


M 


44 


(- 1) 


D 


45 


N 


41 


(+ I) 


I* 


40 


I 


37 


(- 3) 


N* 


40 


F 


37 


(- 8) 


0* 


37 


0 37 


(N.C.) 



* Units participated in THAs pilot 1985-87 



for Match Units 





1988 




C 62 


( + 


0 


;+ 3) 


B 67 




1 
1 


;+ 2) 


E 5b 


f X 

I T 


1 


;+ 1) 


G 53 




0 

c 


;- 2) 


A 53 


( ^ 


A 


;N.C.) 


K 51 




A 
H 


; +2) 


H 51 


( 4. 
1 1 


1 i 


,N.C.) 


J 49 




4, 


3) 


L 47 




5; 


2) 


M 46 


( + 


2; 


.+ 1) 


0 46 


( + 


9; 


.+ 9) 


N 45 


( + 


4; 


+ 5) 


D 45 




3; 


N.C.) 


F 44 


( + 


7; 


- 1) 


I 44 


( + 


7; 


+ 4) 
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TABLE 12 

Range and Decile Distributions of Match Units Per Year 

Range Deciles : Units 

1986 37 - 59 (22 points) 30th: 1; 40th: 7; 50th: 7; 60th: 0; 7pth: 0 

1SS7 37 - 58 (21 points) 30th: 3; 40th: 4; 50th: 8; 60th: 0; 70th: 0 

1988 44 - 62 (18 points) 30th: 0; 40th: 8; 50th: 6; 60th: 1; 70th: 0 
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TABLE 13 

Overview of Change- in 15 Match Units and 16 CDP Units 

Grade ^-v^i'a^e 
Unit\ 3 6 8 L'nit\ 



A 






NC 


Perquiirans 


+ 


+ 


NC 






+ 




Chowan 


+ 


+ 


+ 


c 


+ 




+ 


N. Hanover 


+ 


+ 




D 






NC 


Greene 


+ 


a. 


- 


E 


+ 




+ 


Harnett^ 


- 


H 




F 








Rapids 


- 


- 


NC 


6 


_ 






Tarboro 








H 




+ 


NC 


Montgorr.ery 


+ 




NC 


I 






+ 


Salisbury 


+ 


"V 


+ 


J 








Mecklenburg 


- 


- 


+ 


K 




+ 


+ 


Alexander 


+ 


NC 


+ 


L 








Burke 


+ 


+ 


+ 


LI 

W 


1 

T 


MP 




Oranae 


+ 




+ 


N 


+ 




+ 


Burlington 


+ 


+ 


+ 


0 


+ 


+ 


+ 


Haywood 


MC 




+ 










Buncombe 


+ 


+ 




Total 


8+ 




8+ 












7- 


8- 


4- 


Total 






1C+ 




ONC 


2NC 


3NC 




3- 


4- 


3- 












INC 


INf 


3NC 



+ = improved in 1988 over 1986 
- = declined in 1988 over 1986 
NC = No change in 1988 over 1986 



When ii IS remembered that the match units were selected because of their 
resemblance to the CDP units on critical characteristics, the markedly different 
patterns are difficult to reconcile, unless the Career Development Program effect 
is taken into account* While it would be foolhardy to attribute all of the 
differences to CDP, it would be equally foolish to contend that CDP made no 
dif ference>,. 

Clearly, patterns of improved student performance have been demonstrated in 
the CDP units both as a group and in many individual units. Moreover, the growth 
seems as likely to occur among high-performing units as in middling or low-per- 
forming units. This improvement appears to be as likely in 3rd or 6th grade as 
8th grade. By contrast, we see no clearly defined patterns among the individual 
match urjits, although it could be argued that, as a group, the match units have 
.experienced some improvement. This, of course, would reflect the general state- 
wide improvement in CAT scores. This generalized improvement, however, is quite 
inconsistently demonstrated in the individual units, in contrast with the individ- 
ual CDP units. 

Student to Student Comparison 

Before concluding the presentation of the CDP unit data in contrast to the 
match unit data, one more table should be presented. Table 14 presents the 
achievement data for 6th grade students in 1986 and the same data for the same 
students two years later, that is, 8th grade in 1988 for both groups of units. 

In 1963, John B. Carroll published one of the most influential articles in 
the educational literature. (Carroll, 1963). In "A Model of School Learning", 
Carroll posited that students would learn as a function of aptitude, perseverance, 
opportunity, ability to learn, and quality of instruction. 
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While no one of these fivciors could overcome the others, the interaction was not 
clear, but obviously students will differ in aptitude and perseverance. 
Opportunity and quality of instruction, however, are more or less controllable 
by the institution. We would assume, then, that scores changes in the s^ine 
^ population, over time, coula refU;ct a change in opportunity to learn anc'/or 
quality of instruction. That is, if we compared scores of a student over time, 
some part of the diiference should be attributable to quality of instruction. 

Table 14 provides us with the opportunity to test the hypothesis with the 
units we have been considering. This table shows CAT scores (unit averages) 
for students in Grade 6 in 1986. It also shows CAT scores for essentially 
these same students two years later (1988) when they were in Grade 8. What do 
the data shew? 

First, let us consider only the CDP units. In 198G, five units (indicated 
by asterisk) .had CAT averages below the 50th percentile. In 1988, only two 
units were below the 50th, and one of these was at 49th. Students in three 
units, then, had improved their scores enough to propel themselves ever this 
line of demarcation. The neediest students, in short, performed better. I'r- 
fcrtunately, this pattern of improvement was not universal in the CDP units. 
Eight units declined between 6th c.nd 8th grade. However, these losses were, by 
and large modest. Four units lost one percentile point, v;hile three lost two points. 
One unit lost six points. 

However, among the eight gaining units, the gains were significant. The 
smallest gain was three percentile pointr, (experieriLod by four units), while 
other gains of tour, five, six, tind bixtei.-n points were gained: 



TABLE 14 



Coiiiparison of CAT Scores for 6th Grade (8G) and 
8th Grade (88) for CDP and Katch Units 



IK IT 


6th Gr(86) 


8th Gr(88) 


Diff 


UNIT 


6th Gr(86) 


8th Gr(88) 


Diff 


Burl ington 


69 


68 


-1 


C 


63 


62 


-1 


Kaywood 


63 


62 


-1 


A 


63 


53 


-10 


Koanoke Rapids 


62 


65 


+3 


L 


56 


47* 


-9 


Burke 


59 


53 


-6 


B 


56 


57 


+1 


K. hencver 


58 


57 


-1 


J 


55 


49* 


-6 


Buncombe 


58 


56 


-2 


E 


54 


55 


+1 


Mecklenburg 


57 


55 


-2 


G 


52 


53 


+1 


Alexander 


57 


55 


Im 


c 

r 


HO 




~H 


Chov.-an 


54 


57 


+3 


D 


48* 


45* 


-3 


Orange 


53 


52 


-1 


M 


47* 


46* 


-1 


Harnett 


50 


53 


+3 


H 


46* 


51 


+5 


TarL/Oro 


48* 


52 


+4 


N 


43* 


45* 


+2 


Ferquimans 


45* 


51 


+6 


K 


42* 


51 


+9 


Kontgoinery 


44* 


49* 


45 


I 


39* 


44* 


+5 


Salisbury 


38* 


54 


+16 


0 


37* 


46* 


+9 


Greene 


33* 


36* 


+3 











* = Unit average below 50th percentile 
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Points 


-6 


-5 


-4 


-3 


-2 


-1 


0 


+1 


+2 


+3 


H4 


+5 


+6 


+16 


Units 


1 


0 


0 


0 


3 


4 


0 


0 


0 


4 


1 


] 


1 


1 



In short, the gainers improved more than the losers lost. Moreover, among 
the lower half of the units (arranged by 6th grade scores) only one lost grouna. 
Children who had done well in 6th grade continued to do well and children who 
had done relatively poorly in 6th grade made remarkable growth. 

Now, let us turn to the right side of the table. The picture here is some- 
what different. Notice that in 1986, eight units {F, D, M, H, N, K, I, and 0) 
oil scored below the 50th percentile, in Grade 6. In 1988, eight units were 
still below the 50th percentile. However, only Units H and K had moved from 
below to above the 50th percentile. Units F, D, M, N, I, and 0 were still below 
the median, although three of them (N, I, ana 0) had improved. Unfortunately, 
F, D, and M had actually declined further. More unfortunc»tely , Units L and J had 
also fallen below the median, going from 56th and b5th to 47th and 49th respec- 
tively. While the very neediest students had improved their scores, in some cases 
dramatically, many of the "near needy" had declined markedly. The trend for 
marginal units (those near, but below the median) was clearly down from tht edian 

Moreover, among the seven top-scoring units in 1988, (the only ones above the 
median in 1986), four had lost ground as compared to themselves when in Grade 6. 
The losses were not small. While the highest scoring district had lost only 1 
point, the other units lost 10, 9, and 6 percentile points. 
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The only gains in the top seven were one percentile point each in three units: 



Points 


i 

-10 


i — 
-9 


-8 


-7 


-6 


-5 


~4 


-3 


-2 


-1 


0 


+1 


+2 


+3 


+4 


+5 


+6 


+7 


+8 


+9 


+10 


Units 


1 


1 


0 


0 


i 


0 


1 


1 


0 


2 


0 


3 


1 


p 


0 


2 


0 


0 


0 


2 


0 



It. must not be overlooked that the five lowest-scoring sixth grades all 
made improvement, repeating the pattern seen among CDP's lowest scoring units. 
Their growth, however, was not as rapid or as large as the growth of the CDP 
units, while the losing units tended to lose more than the losing CDP units. 
Clearly, the comparison favors CDP, indicating that sustained growth occurred 
in the majority of these units with relatively modest losses. 

Considering all of the data presented, in Tables 1-14, what conclusions 
may be drawn? These data show that: 

0 among CDP units, in Grades 3, 6. and 8, the number scoring 
below the 50th percentile declined over the period 1986-88. 
In I98&, only 1 unit was below the median in Grade 3, 3 in 
Grade 6, and 2 in Grade 8. By contrast, 3 of the match units 
were below the 50th percentile, down from 4. Nine units were 
below the median in 6th grade in 1988, up from 8. In 8th grade 
there was no change in absolute numbers, with 8 units below the 
national average. 
0 In Grades 3 and 6, CDP units raised the floor from the 43rd and 
:53rd percentiles to the 47th and 4?nd percentile, respectively. 



While the floor for 8th grade fell from the 40th to tho 36lh 

pe^^centile, if we discount a single unit, the floor stayed 

steady at the 49th percentile over the period. 

In the match units, the floor rose in Grades 3 and 8, but fell 

in Grade 6, with two units experiencing declines. 

Similarly, the ceilings rose in each grade among CDP units, but 

rose in the match units' 3rd and 8th grades. 

• More than 63% of the CDP units experienced growth in each grade 
over the period. Between 33% and 50% of the match units improved 
at each grade level. 

• Decreases of scores over the period was much more likely to occur 
among units in the match set. 

• If we compare 1986 -6 th grades to 1988 8th grades, we see the largest 
gains among the neediest uni.ts. However, units of the match set were 
more likely to lose, and to lose more, than units in the CDP set. 

• Finally, no CDP unit experienced declines In all three grades 
over the period. In the match units, two units experienced losses 
in each grade. Among the CDP set, five units made gains in each 
grade. In the match set, only two units equalled their achievement. 

CDP Units Within Regions 

One more type of analysis may Increase our understanding of the performance 
of students in Career Development Program pilot units. It will be interesting to 
compare the units with the performance of the units comprising the educational 
regions in which they fall. In this way, we will be able to look at the CDP 
units In the context of their region to see whether patterns of improvement of 
student achievement appear. 
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Table 15 a, b, and c present a picture of 3rd grade achievement in all eight 
educational regions. The letters *'x" or "o" have been arbitrarily assigned 
to the CnP pilots in each region. While the interaction of the three tables con- 
tains much interesting information, we note only some of this. Generally, the 
performance improved in all regions over the period in question. However, the 
relative position of n.any of the pilot units, especially if they were on the lower 
side, improves more rapidly. Since the line represents the continuum of perform- 
arce in the region, careful attention should be paid to movement of the end bars 
and to the x's and o's. Obviously, at 3rd grade, the CDP units improved absolutely 
and relatively. While some of the units moved more rapidly than others, most moved 
to the right of the line, the desired direction. 

Tables 16 a, b, and c present the 6th grade data in the same format. It is 
interesting to note that, over time, the two units in the same region might ex- 
change places, but genera'lly both continue moving right. Presumably, this indicates 
accelerated rates of change in one unit compared to another. This phenomenon is 
most clearly seen in Regions VII and VIII. 

Finally, Tables 17 a, b, and c present information about 8th grade performance 
in the units and regions from 1985-88. Some change in the ranges, in both direc- 
tions, is visible indicating that some units are improving scores, while others 
are declining. However, the general upv/ard movement pattern of the CDP units, 
individually and as a group, remains distinct. 

Thus, compared with their regions, the CDP units students generally performed 
so that their improvements were marked, relative to others in the region. The 
pattern holds over time at all grade levels. 
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TABLE 15 a 
1986: 3rd Grade CAT % lies 



REGIONS/CDP V 

"HOWAN: 43 PERQ: 48 
^ GREENE: 47 N. HANOVER: 61 
R. RAPIDS: 67 TARBOHO: 61 

. MONn 52 HARNETT: 64 

ORANGE: 59 BURL: 76 

- SAUS: 44 MECK: 60 

ALEX: 59 BURKE: 64 

• BUNC: 67 HAYWD: 71 



1 1 1 1 I 1 1 I I 



0 H I 



H 0 1 



0 i 



M O 



K O I 



10 20 30 40 50 60 70 80 90 100 
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TABLE 15 b 
1987: 3rd Grade CAT % iles 



REGIONS/CDP UNITS 

I.CHOWAN: 39 PEP5: 62 

y- GREENE: 46 N. IIANOVER: 67 

III • a RAPIDS: 68 TARBORO: 70 

IV- 'MOOT^ 59 HARNEm 56 

V- ORANGE: 57 BURL: 73 

VI- SAUS: 52 MECK: 57 
VII 'ALEX: 59 BURKE: 63 
VIII • BUNC: 65 HAYWD: 68 



Hi- 



•e—i 



9 0 I 

H 0 1 



■f-f 



10 20 30 40 SO 60 70 80 90 100 
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TABLE 15 c 
1988: 3rd Grade CAT % lies 



REGIONS/CDP UNITS 

I -CHOWAN: SO f^EBQ: 70 |_ 



II -GREENE: 55 N.HANOVER: 68 



ni • R RAPIDS: 63 TARBORO: 67 



IV -MONT: 56 HARNETT: 60 



V- ORANGE: 62 BURL: 77 



VI-SAUS: 47 MECK: 58 



VII -ALEX: 71 BURKE: 69 



VIII -BUNC: 68 HAYWD: 71 



•a- 



■ac-d 



^ — I 



- I- 



OH I 



K O \ 



10 20 30 40 60 60 70 80 90 100 
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TABLE 16 a 
1986; 6th Grade CAT % lies 



REGIONS /CDPIINITS 

I -CHOWAN; 54 PERQ: 45 



HI - R RAPIDS: 62 TARBORO: 48 



IV -MONT: 44 HARNETT: 50 



V- ORANGE: 53 BURL: 69 



VI - SAUS: 38 MECK: 57 



VII -ALEX: 57 BURKE: 59 



VUI-BUNC: 58 HAYWD: 63 



n-vCRES.'E: .33 N. HANOVER: 58 



I 3f- 



1 1 I I I I I I I I 



■« 1 



H O I 



10 20 30 40 60 



I I I I I I I I I - f 



60 70 



80 90 100 
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TABLE/ 16 1) 
1987: 6th Grade CAT % iles 



REGIONS/CDP UNITS 

:H0WAN: 49 PERQ: 53 
.: GREENE: 39 N.HANOVER: 62 

•• R. RAPIDS: 65 TARBORO: SO 

.V-MONTi SO HARNETT: 49 

ORANGE: 54 BURL: 72 

• " - SAUS: 54 MECK: 55 

•/■:;-A'JEX: 58 BURKE: 57 

■/ill - BUNC: 61 HAYWD: 66 



I M l O 



OH I 



Jl o I 



10 20 30 40 50 60 70 80 90 100 
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TABLE 16 C 
1988: 6th Grade CAT % lies 



RSGIONS/CDP UNITS 

:H0WAN: 59 PERQ: 50 



i Q HI 



:;reene: 42 N. HANOVER: 66 



RAPIDS: 56 TARBORO; 49 



MONTi 50 HARNETT: 51 



ORANGE: 52 BURL: 71 



SAUS: 42 MECK: 56 



V.': - ALEX: 57 - BURKE: 62 



VUl • BUNC: 63 HAYWD: 62 



I »■ 



1 



OH i 



I I I I I I I I I I 



10 20 30 40 50 60 70 80 90 100 
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TABLE 17 a 
1986: 8th Grade CAT % iles 



REGIONS/CDP UNITS 

"HOWAN: 50 PERQ: 51 
' OREENE: 40 N. HANOVER: 51 
R. RAPIDS: 65 TARBORO: 54 

' • MONT: 49 HARNETT: 49 

ORANGE: 49 BURL: 64 

" SAUS: 53 MECK: 51 

^ • ALEX: 52 BURKE: 50 

• BUNC: 58 HAYWD: 61 
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TABLE 17 b 
1987: 8th Grade CAT % iles 



I vLGIONS/CDP UNIT$ 

CHOWAN: 47 PERQ; 53 
' GREENE; 38 N. HANOVER: 55 

R. RAPIDS: 61 TARBORO: 49 

• MONTi 52 HARNETT: 51 |. 
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SAUS: 45 MECK: 51 



AJ-EX: 49 BURKE: 51 



BUNG: 62 HAYWD: 62 
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TABLE 17 c 
1988: 8th Grade CAT % lies 



3aGIONS/CDP ITVIT5S 

CHOWAN: 57 PERQ: 51 



•IREENE: 36 N.HANOVER: 57 



•.I RAPIDS: 65 TARBORO: 52 



MONT: 49 HARNETT: 53 



•ORANGE: 52 BURL: 68 



5AUS: 54 MECK: 55 



: - ALEX: 55 BURKE: 53 



BUNC: 56 HAYV/D: 62 
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CONCLUSION 



It is clear from the preceding analysis that, as Larry Lazotte has said, 
school urn'ts are either improving or declining. When student achievement in the 
Career Development Program pilot units is examined the general tendency is towards 
improvement. Fewer CDP units scored below the national median fur third grade in 
1988 than did in 1986. The same was true for 6th and 8th grade achievement. In 
the group of match units, we did see a decline of units scoring below the median 
in gradt- 3, but we saw no improvement on this parameter in Grades 6 or 8. 

In general, CDP units, for the most part, experienced improvement over the 
period in Grades 3, 6, and 8. In Grade 3, 13 CDP units improved or showed no 
change. In Grade 6, 12 of the CDP units improved or showed no change. In Grade 8, 
13 CDP units iipproved or showed no change. Tlie corresponding numbers for the match 
group were eight units for Grade 3, seven units for Grade 6, and ten for Grade 8. 

If we focus our attention on the four lowest performing units in each group, 
we find that, in the CDP units, the lowest performers all made improvements in 
Grades 3 and 6, whiles one unit gained, one lost, and two stayed the same in Grade 
8. By contrast, the match set, only two units improved in Grade 3 over the 
period, while two lost. In Grade 6, one unit improved while three lost. In Grade 
8, two units improved, one remained unchanged, and one unit lost. 

At the upper end of the scale, the two groups were more similar. In 3rd 
grade, three of the top four CDP units improv(?d and one unit remained unchanged, I 
the match group, for third grade, three units improved, while one lost ground. In 
Graae 6, all four top performing CDP units made improvement, while two of the 
match units improved, one remained unchanged, and one lost. In Grade 8, we see 
exactly the same pattern in both groups that were seen in Grade 3, 
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As Table 13 shows, improveinerit was more likely to occur in the CDP units than 
in the matched sot. It we think of each grade per unit us a iiieasureiiiLMit. opportunity, 
the CDP units as a group improved on 695^ of the measures, stayed unchcinned on 
10%, and declined on 21% of the measures. The matches' corresponding rates were 
47% improvements, 11% no change, and 42% decline. Whereas clear patterns of im- 
provement are demonstrated by the CDP units, we would characterize the change in 
the match units as random improvement or diffused change. While it is unreasonable 
to credit all the improvement experienced in the CDP units to their efforts in the 
pilot, certainly some of the improvement should be credited to the pilot. It is 
important to remember that these analyses were undertaken precisely because of the 
almost universal sentiment among CDP participants that instruction had improved 
because of the unit's participation in the pilot. 

Perhaps the most interesting demonstration of CDF ' s. efficacy is presented in 
Table 14. Remember that these scores were posted by largely the same population 
at the beginning of the CDP experience and two years later. Both the direction, 
location, and amount of improvement are heartening. All of the units scoring 
below the national median in 1986 showed sizeable improvements. While the de- 
creases in the high-and middle-scoring units is cause for concern, the relatively 
small decreases, with one exception, do not obviate the benefits of the CDP. 

By contrast, in the match units, while the lowest performing units do make 
commendable gaini, the relatively larger, more wide-spread losses among the other 
units is noteworthy. Again, the random nature of the change offers confirmation 
that quite different conditions obtained in this group from the CDP group. 
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It has been seriously suggested tliat students would suffer because of a 
perceive/l necessity for "robotized" teaching in the CDP units. These data put 
the lie to that charge. Clearly, students in the CDPs, taken as a group, have 
benefitujd from more wide-spread goJd teaching than did students in the match 
sample. While we cannot factor out how much of the achievement gains to credit 
with specific feature of Career Development or other innovations, for that matter, 
it is clear that the sixteen participating units have posted sigrificant gains in 
student achievement, in both relative and absolute terms. 
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